network namespace
我们在使用 openstack 的时候，会用到ip netns命令进到docker的network namespace中抓包排查问题，在k8s中我们也可以使用类似的方式，不过要使用 ip netns 稍微复杂点，得手动连接容器的 ns id到/var/run/netns下（没有自动）1
2
3
4
5
6
7pid=$(docker inspect -f '{{.State.Pid}}' ${container_id})
mkdir -p /var/run/netns/
ln -sfT /proc/$pid/ns/net /var/run/netns/$container_id
ip netns exec "${container_id}" ip -s link show eth0
不过我们可以使用nsenter命令，pod内抓包排查问题基本思路：
- pod副本数缩减为1
查看pod id
1
kubectl -n <namespace> describe pod <pod> | grep -A10 "^Containers:" | grep -Eo 'docker://.*$' | head -n 1 | sed 's/docker:\/\/\(.*\)$/\1/'
获得容器进程的 pid
1
docker inspect -f {{.State.Pid}} <container>
进入容器的network namespace
1
nsenter -n --target <PID>
使用tcpdump等工具抓包分析
some namespaces
通常连接Docker容器并与其进行交互有四种方法。详情见：Docker Enter Demo，下面摘录nsenter连接的方式。
查看是否安装nsenter
[root@localhost ~]# whereis nsenter
nsenter: /usr/bin/nsenter /usr/share/man/man1/nsenter.1.gz
如果没安装可创建install.sh，并执行
1 | #!/bin/bash |
方式一：创建docker-enter并置于$PATH下1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35#!/bin/sh
if [ -e $(dirname "$0")/nsenter ]; then
# with boot2docker, nsenter is not in the PATH but it is in the same folder
NSENTER=$(dirname "$0")/nsenter
else
NSENTER=nsenter
fi
if [ -z "$1" ]; then
echo "Usage: `basename "$0"` CONTAINER [COMMAND [ARG]...]"
echo ""
echo "Enters the Docker CONTAINER and executes the specified COMMAND."
echo "If COMMAND is not specified, runs an interactive shell in CONTAINER."
else
PID=$(docker inspect --format "{{.State.Pid}}" "$1")
if [ -z "$PID" ]; then
exit 1
fi
shift
OPTS="--target $PID --mount --uts --ipc --net --pid --"
if [ -z "$1" ]; then
# No command given.
# Use su to clear all host environment variables except for TERM,
# initialize the environment variables HOME, SHELL, USER, LOGNAME, PATH,
# and start a login shell.
#"$NSENTER" $OPTS su - root
"$NSENTER" $OPTS /bin/su - root
else
# Use env to clear all host environment variables.
"$NSENTER" $OPTS env --ignore-environment -- "$@"
fi
fi
常见问题：nsenter: failed to execute su: No such file or directory
这是由于容器中的PATH 路径问题，使用/bin/su 即可。
方式二：也可以将其放在.bashrc中，就可以方便的使用了。（运行source ./bashrc不重启生效）1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43#docker
#export DOCKER_HOST=tcp://localhost:4243
alias docker-pid="sudo docker inspect --format '{{.State.Pid}}'"
alias docker-ip="sudo docker inspect --format '{{ .NetworkSettings.IPAddress }}'"
#the implementation refs from https://github.com/jpetazzo/nsenter/blob/master/docker-enter
function docker-enter() {
if [ -e $(dirname "$0")/nsenter ]; then
# with boot2docker, nsenter is not in the PATH but it is in the same folder
NSENTER=$(dirname "$0")/nsenter
else
NSENTER=nsenter
fi
[ -z "$NSENTER" ] && echo "WARN Cannot find nsenter" && return
if [ -z "$1" ]; then
echo "Usage: `basename "$0"` CONTAINER [COMMAND [ARG]...]"
echo ""
echo "Enters the Docker CONTAINER and executes the specified COMMAND."
echo "If COMMAND is not specified, runs an interactive shell in CONTAINER."
else
PID=$(sudo docker inspect --format "{{.State.Pid}}" "$1")
if [ -z "$PID" ]; then
echo "WARN Cannot find the given container"
return
fi
shift
OPTS="--target $PID --mount --uts --ipc --net --pid"
if [ -z "$1" ]; then
# No command given.
# Use su to clear all host environment variables except for TERM,
# initialize the environment variables HOME, SHELL, USER, LOGNAME, PATH,
# and start a login shell.
#sudo $NSENTER "$OPTS" su - root
sudo $NSENTER --target $PID --mount --uts --ipc --net --pid su - root
else
# Use env to clear all host environment variables.
sudo $NSENTER --target $PID --mount --uts --ipc --net --pid env -i $@
fi
fi
}
执行：source ./bashrc，让修改生效。
进入容器：
docker-enter 容器ID
参考：